Model assessment plots for multilevel logistic regression

This paper extends the Bayes marginal model plot (BMMP) model assessment technique from a traditional logistic regression setting to a multilevel application in the area of criminal justice. Convicted felons in the United States receive either a prison sentence or a less severe jail or non-custodial sentence. Researchers have identified many determinants of sentencing variation across the country, some individual such as type of crime and race, and some based on geographical units such as county crime rate. Multilevel rather than conventional regression should be used to quantify any interplay between such individual-level and county-level effects since the covariates have a hierarchical structure. Questions arise, however, as to whether a multilevel model provides an adequate fit to the data, and whether the computational burden of a multilevel model over a conventional model is justified. Residual plots, traditionally used to assess regression models, are difficult to interpret with a binary response variable and multilevel covariates, as in this case. BMMPs, an alternative graphical technique, can be used to visualize goodness of fit in such settings. The plots clearly demonstrate the need to use multilevel modeling when analyzing data such as these.